169 research outputs found
One-class classifiers based on entropic spanning graphs
One-class classifiers offer valuable tools to assess the presence of outliers
in data. In this paper, we propose a design methodology for one-class
classifiers based on entropic spanning graphs. Our approach takes into account
the possibility to process also non-numeric data by means of an embedding
procedure. The spanning graph is learned on the embedded input data and the
outcoming partition of vertices defines the classifier. The final partition is
derived by exploiting a criterion based on mutual information minimization.
Here, we compute the mutual information by using a convenient formulation
provided in terms of the -Jensen difference. Once training is
completed, in order to associate a confidence level with the classifier
decision, a graph-based fuzzy model is constructed. The fuzzification process
is based only on topological information of the vertices of the entropic
spanning graph. As such, the proposed one-class classifier is suitable also for
data characterized by complex geometric structures. We provide experiments on
well-known benchmarks containing both feature vectors and labeled graphs. In
addition, we apply the method to the protein solubility recognition problem by
considering several representations for the input samples. Experimental results
demonstrate the effectiveness and versatility of the proposed method with
respect to other state-of-the-art approaches.Comment: Extended and revised version of the paper "One-Class Classification
Through Mutual Information Minimization" presented at the 2016 IEEE IJCNN,
Vancouver, Canad
Intelligence for embedded systems: A methodological approach
Addressing current issues of which any engineer or computer scientist should be aware, this monograph is a response to the need to adopt a new computational paradigm as the methodological basis for designing pervasive embedded systems with sensor capabilities. The requirements of this paradigm are to control complexity, to limit cost and energy consumption, and to provide adaptation and cognition abilities allowing the embedded system to interact proactively with the real world. The quest for such intelligence requires the formalization of a new generation of intelligent systems able to exploit advances in digital architectures and in sensing technologies. The book sheds light on the theory behind intelligence for embedded systems with specific focus on: ·       robustness (the robustness of a computational flow and its evaluation); ·       intelligence (how to mimic the adaptation and cognition abilities of the human brain), ·       the capacity to learn in non-stationary and evolving environments by detecting changes and reacting accordingly; and ·       a new paradigm that, by accepting results that are correct in probability, allows the complexity of the embedded application the be kept under control. Theories, concepts and methods are provided to motivate researchers in this exciting and timely interdisciplinary area. Applications such as porting a neural network from a high-precision platform to a digital embedded system and evaluating its robustness level are described. Examples show how the methodology introduced can be adopted in the case of cyber-physical systems to manage the interaction between embedded devices and physical world.. Researchers and graduate students in computer science and various engineering-related disciplines will find the methods and approaches propounded in Intelligence for Embedded Systems of great interest. The book will also be an important resource for practitioners working on embedded systems and applications
Anomaly and Change Detection in Graph Streams through Constant-Curvature Manifold Embeddings
Mapping complex input data into suitable lower dimensional manifolds is a
common procedure in machine learning. This step is beneficial mainly for two
reasons: (1) it reduces the data dimensionality and (2) it provides a new data
representation possibly characterised by convenient geometric properties.
Euclidean spaces are by far the most widely used embedding spaces, thanks to
their well-understood structure and large availability of consolidated
inference methods. However, recent research demonstrated that many types of
complex data (e.g., those represented as graphs) are actually better described
by non-Euclidean geometries. Here, we investigate how embedding graphs on
constant-curvature manifolds (hyper-spherical and hyperbolic manifolds) impacts
on the ability to detect changes in sequences of attributed graphs. The
proposed methodology consists in embedding graphs into a geometric space and
perform change detection there by means of conventional methods for numerical
streams. The curvature of the space is a parameter that we learn to reproduce
the geometry of the original application-dependent graph space. Preliminary
experimental results show the potential capability of representing graphs by
means of curved manifold, in particular for change and anomaly detection
problems.Comment: To be published in IEEE IJCNN 201
Change Point Methods on a Sequence of Graphs
Given a finite sequence of graphs, e.g., coming from technological,
biological, and social networks, the paper proposes a methodology to identify
possible changes in stationarity in the stochastic process generating the
graphs. In order to cover a large class of applications, we consider the
general family of attributed graphs where both topology (number of vertexes and
edge configuration) and related attributes are allowed to change also in the
stationary case. Novel Change Point Methods (CPMs) are proposed, that (i) map
graphs into a vector domain; (ii) apply a suitable statistical test in the
vector space; (iii) detect the change --if any-- according to a confidence
level and provide an estimate for its time occurrence. Two specific
multivariate CPMs have been designed: one that detects shifts in the
distribution mean, the other addressing generic changes affecting the
distribution. We ground our proposal with theoretical results showing how to
relate the inference attained in the numerical vector space to the graph
domain, and vice versa. We also show how to extend the methodology for handling
multiple change points in the same sequence. Finally, the proposed CPMs have
been validated on real data sets coming from epileptic-seizure detection
problems and on labeled data sets for graph classification. Results show the
effectiveness of what proposed in relevant application scenarios
Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
Among the various architectures of Recurrent Neural Networks, Echo State
Networks (ESNs) emerged due to their simplified and inexpensive training
procedure. These networks are known to be sensitive to the setting of
hyper-parameters, which critically affect their behaviour. Results show that
their performance is usually maximized in a narrow region of hyper-parameter
space called edge of chaos. Finding such a region requires searching in
hyper-parameter space in a sensible way: hyper-parameter configurations
marginally outside such a region might yield networks exhibiting fully
developed chaos, hence producing unreliable computations. The performance gain
due to optimizing hyper-parameters can be studied by considering the
memory--nonlinearity trade-off, i.e., the fact that increasing the nonlinear
behavior of the network degrades its ability to remember past inputs, and
vice-versa. In this paper, we propose a model of ESNs that eliminates critical
dependence on hyper-parameters, resulting in networks that provably cannot
enter a chaotic regime and, at the same time, denotes nonlinear behaviour in
phase space characterised by a large memory of past inputs, comparable to the
one of linear networks. Our contribution is supported by experiments
corroborating our theoretical findings, showing that the proposed model
displays dynamics that are rich-enough to approximate many common nonlinear
systems used for benchmarking
A characterization of the Edge of Criticality in Binary Echo State Networks
Echo State Networks (ESNs) are simplified recurrent neural network models
composed of a reservoir and a linear, trainable readout layer. The reservoir is
tunable by some hyper-parameters that control the network behaviour. ESNs are
known to be effective in solving tasks when configured on a region in
(hyper-)parameter space called \emph{Edge of Criticality} (EoC), where the
system is maximally sensitive to perturbations hence affecting its behaviour.
In this paper, we propose binary ESNs, which are architecturally equivalent to
standard ESNs but consider binary activation functions and binary recurrent
weights. For these networks, we derive a closed-form expression for the EoC in
the autonomous case and perform simulations in order to assess their behavior
in the case of noisy neurons and in the presence of a signal. We propose a
theoretical explanation for the fact that the variance of the input plays a
major role in characterizing the EoC
Change Detection in Multivariate Datastreams: Likelihood and Detectability Loss
We address the problem of detecting changes in multivariate datastreams, and
we investigate the intrinsic difficulty that change-detection methods have to
face when the data dimension scales. In particular, we consider a general
approach where changes are detected by comparing the distribution of the
log-likelihood of the datastream over different time windows. Despite the fact
that this approach constitutes the frame of several change-detection methods,
its effectiveness when data dimension scales has never been investigated, which
is indeed the goal of our paper. We show that the magnitude of the change can
be naturally measured by the symmetric Kullback-Leibler divergence between the
pre- and post-change distributions, and that the detectability of a change of a
given magnitude worsens when the data dimension increases. This problem, which
we refer to as \emph{detectability loss}, is due to the linear relationship
between the variance of the log-likelihood and the data dimension. We
analytically derive the detectability loss on Gaussian-distributed datastreams,
and empirically demonstrate that this problem holds also on real-world datasets
and that can be harmful even at low data-dimensions (say, 10)
- …